System and method for selecting a two-dimensional region of interest using a range sensor

ABSTRACT

A system includes a first camera defining a first camera coordinate system (C 1 ) and configured to acquire a first image of a scene. A range sensor is spaced a first distance from the first camera and defines a range sensor coordinate system (R). A controller is operatively connected to the first camera and range sensor. The controller has a processor and a tangible, non-transitory memory device on which is recorded instructions for executing a method for obtaining a two-dimensional region of interest (u 1 *, v 1 *) in the first image, which is a two-dimensional intensity image. The first image is represented by a plurality of first points (u 1 , v 1 ) in a first image plane. The controller is configured to acquire a range image of the scene with the range sensor. The range image is represented by a plurality of second points (u 2 , v 2 , d) in a second image plane.

TECHNICAL FIELD

The disclosure relates generally to a system and method for selecting atwo-dimensional region of interest in a two-dimensional image using arange sensor.

BACKGROUND

Two-dimensional images obtained from conventional digital cameras mayinclude distracting and irrelevant clutter, for example, in thebackground or other parts of the image. Typically, segmentation of thetwo-dimensional image may involve physical modification of theenvironment, such as adding a curtain to remove irrelevant backgroundclutter. Electronic segmentation of the two-dimensional image using onlytwo-dimensional cues may be time-consuming.

SUMMARY

A system includes a first camera defining a first camera coordinatesystem (C₁) and configured to acquire a first image of a scene. A rangesensor is spaced a first distance from the first camera and defines arange sensor coordinate system (R). For example, the lens of the firstcamera and the sensor portion of the range sensor may be positioned atthe origins of the first camera coordinate system (C₁) and range sensorcoordinate system (R), respectively.

A controller is operatively connected to the first camera and rangesensor. The first camera, the controller and the range sensor may bepart of a single device. The controller has a processor and a tangible,non-transitory memory device on which is recorded instructions forexecuting a method for obtaining a two-dimensional region of interest(u₁*, v₁*) in the first image, which is a two-dimensional intensityimage.

Execution of the instructions by the processor causes the controller toacquire a first image of the scene with the first camera. The firstimage is represented by a plurality of first points (u₁, v₁) in a firstimage plane. The controller is configured to acquire a range image ofthe scene with the range sensor. The range image is represented by aplurality of second points (u₂, v₂, d) in a second image plane. Each ofthe plurality of second points (u₂, v₂, d) in the range image includes arange distance (d) corresponding to a respective distance from the rangesensor to the objects in the scene.

The controller is configured to convert the range image to athree-dimensional sample of points (x₂, y₂, z₂) in the range sensorcoordinate system (R); and select a three-dimensional region of interest(x₂*, y₂*, z₂*) in the range sensor coordinate system (R) from thethree-dimensional sample of points (x₂, y₂, z₂). The selectedthree-dimensional region of interest (x₂*, y₂*, z₂*) in the range sensorcoordinate system (R) may include only objects in the scene that areless than a minimum distance from the range sensor. A spatial locationof each of the selected points in the three-dimensional volume may be afunction of the range distance (d). The selected three-dimensionalregion of interest (x₂*, y₂*, z₂*) in the range sensor coordinate system(R) may include only objects in the scene within or on the surface of athree-dimensional volume. A spatial location of each of the selectedpoints in the three-dimensional volume may be a function of time suchthat the position, size or shape of the three-dimensional volume maychange over time.

The controller is configured to transform the three-dimensional regionof interest from the range sensor coordinate system (R) to the firstcamera coordinate system (C₁) [(x₂*, y₂*, z₂*) to (x₁*, y₁*, z₁*)]. Thecontroller is configured to map the three-dimensional region of interest(x₁*, y₁*, z₁*) in the first camera coordinate system (C₁) onto thefirst image plane to obtain the two-dimensional region of interest (u₁*,v₁*).

The first camera may be a digital camera. Utilizing a range image withdistance information can provide fast and cost-effective ways to segmenttwo-dimensional images, thus speeding up the process of analyzing thetwo-dimensional images. The system reduces the portion of the image forwhich other more computationally-expensive algorithms are to beperformed, resulting in an overall speedup of vision processing. Thiscan, for example, prevent false matches when searching two-dimensionalintensity images.

The above features and advantages and other features and advantages ofthe present invention are readily apparent from the following detaileddescription of the best modes for carrying out the invention when takenin connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a system having a first camera, rangesensor and a controller;

FIG. 2 is a flow chart of a process implemented by the controller ofFIG. 1 for obtaining a two-dimensional region of interest in atwo-dimensional first image produced by the first camera of FIG. 1; and

FIG. 3A is a schematic diagram of an example first image produced by thefirst camera of FIG. 1 prior to implementation of the process of FIG. 2;and

FIG. 3B is a schematic diagram of the first image of FIG. 3A afterimplementation of the process of FIG. 2.

DETAILED DESCRIPTION

Referring to the Figures, wherein like reference numbers refer to thesame or similar components throughout the several views, FIG. 1illustrates a system 10 having a first camera 12 defining a first cameracoordinate system (C₁). The system 10 may take many different forms andinclude multiple and/or alternate components and facilities. While anexample system 10 is shown in the Figures, the components illustrated inthe Figures are not intended to be limiting. Indeed, additional oralternative components and/or implementations may be used.

Referring to FIG. 1, a range sensor 14 is spaced a first distance 16from the first camera 12 and defines a range sensor coordinate system(R). In one example, the lens 12A of the first camera 12 and the sensorportion 14A of the range sensor 14 may be positioned at the origins ofthe first camera coordinate system (C₁) and range sensor coordinatesystem (R), respectively. The first camera 12 and range sensor 14 may berigidly mounted on a mounting bracket 18 to define a fixed geometric orspatial relationship.

Referring to FIG. 1, the first camera 12 is configured to acquire afirst image 20 of a scene 22. The first image 20 is represented by aplurality of first points 24 (u₁, v₁) in a first image plane 26. Thefirst image 20 may be a grid of two-dimensional points representing agrayscale or color intensity image. The first camera 12 may be atwo-dimensional digital camera.

Referring to FIG. 1, the range sensor 14 is configured to acquire arange image 30 of the scene. The range image 30 is defined by aplurality of second points 32 (u₂, v₂, d) in a second image plane 34.The range sensor 14 produces a two-dimensional image showing therespective distance, referred to herein as the range distance 36, fromthe sensor portion 14A of the range sensor 14 to each object (such asobject 38) in the scene.

In one example, the range sensor 14 is an infrared time-of-flight sensorwhich resolves distance based on the known speed of light, measuring thetime-of-flight of a light signal between the range sensor 14 and eachpoint in the scene 22. As known to those skilled in the art, the rangedistance 36 (d) may be calibrated using a calibration plate (not shown).The range sensor 14 may be calibrated such that the range distance 36 isgiven directly in physical units, such as feet or meters. The rangesensor 14 may return both a range image and an exactly-registeredinfrared intensity image.

Referring to FIG. 1, the first camera 12 may include at least one lens12A and/or filters (not shown) adapted to receive and/or shape lightfrom the scene 22 onto an image sensor (not shown). The image sensor mayinclude, for example, one or more charge-coupled devices (CCDs)configured to convert light energy into a digital signal. Thecharge-coupled device is an analog device that creates a smallelectrical charge in each photo sensor when impacted by light. Thecharges are converted to voltage one pixel at a time as they are readfrom the chip and turned into digital data using additional circuitry.The image sensor may include a complementary metal-oxide-semiconductorchip (CMOS), which is an active pixel sensor having circuitry next toeach photo sensor converting the light energy to a voltage, which isthen converted to digital data using additional circuitry on the chip.

Referring to FIG. 1, a controller 40 is operatively connected to thefirst camera 12 and range sensor 14. The first camera 12, the controller40 and the range sensor 14 may be part of a single device. Referring toFIG. 1, the controller 40 has a processor 42 and tangible,non-transitory memory 44 on which are recorded instructions forexecuting a method or process 100 for obtaining a two-dimensional regionof interest (u₁*, v₁*) in the first image 20 using the informationobtained by the range sensor 14. The process 100 resides within thecontroller 40 or is otherwise readily executable by the controller 40.Process 100 will be described in detail below with reference to FIG. 2.

Referring to FIG. 1, the controller 40 may include an input device 46and an output device 48 to interact with a user (not shown). The inputdevice 46 may include any device that allows the user to provideinformation or commands to the controller 40. The input device 46 mayinclude, for example, a computer mouse and/or keyboard. The outputdevice 48 may include any device configured to present information tothe user. Thus, the output device 48 may include a display screen orcomputer monitor, such as a liquid crystal display (LCD) screen.

Optionally, a second camera 50 may be operatively connected to thecontroller 40. The second camera 50 may be spaced a second distance 52from the range sensor 14 and rigidly mounted on the mounting bracket 18to define a fixed geometric relationship. The second camera 50 defines asecond camera coordinate system (C₂). The second camera 50 may beconfigured to acquire a third image 54 of the scene 22. The third image54 is defined by a plurality of third points 56 (u₃, v₃) in a thirdimage plane 58. The first and second cameras 12, 50, the controller 40and the range sensor 14 may be part of a single device.

Referring now to FIG. 2, process 100 may begin with step 102 where thecontroller 40 is configured to acquire the first image 20 of the scene22 with the first camera 12. As noted above, the first image 20 isrepresented by a plurality of first points (u₁, v₁) in the first imageplane 26. The steps 102 to 118 of FIG. 2 may be carried out in an orderother than the order described below and some steps may be omitted. Instep 104 of FIG. 2, the controller 40 is configured to acquire the rangeimage 30 of the scene 22 with the range sensor 14.

In step 106 of FIG. 2, the controller 40 is configured to convert therange image 30, represented by the plurality of second points 32 (u₂,v₂, d) in the second image plane 34, to a three-dimensional sample ofpoints (x₂, y₂, z₂) in the range sensor coordinate system (R). Step 106may include a sub-step 106A to obtain a conversion matrix (P₂) forconverting the range sensor coordinate system (R) to the second imageplane 34 such that:

${P_{2}\begin{bmatrix}{x\; 2} \\{y\; 2} \\{z\; 2}\end{bmatrix}} = \begin{bmatrix}{u\; 2} \\{v\; 2} \\d\end{bmatrix}$

The conversion matrix (P₂) may be determined from characteristics of therange sensor 14, such as its focal length. In step 106, an inverse ofthe conversion matrix (P₂) is used to convert the range image 30 (u₂,v₂, d) to the three-dimensional sample of points (x₂, y₂, z₂) such that:

${P_{2}^{- 1}\begin{bmatrix}{u\; 2} \\{v\; 2} \\d\end{bmatrix}} = \begin{bmatrix}{x\; 2} \\{y\; 2} \\{z\; 2}\end{bmatrix}$

In step 108 of FIG. 2, the controller 40 is configured to select athree-dimensional region of interest (x₂*, y₂*, z₂*) from thethree-dimensional sample of points (x₂, y₂, z₂) in the range sensorcoordinate system (R). The selected three-dimensional region of interest(x₂*, y₂*, z₂*) may be dependent on the range distance 36 (d) (i.e. as arange-data-dependent selection volume). For example, thethree-dimensional region of interest may be selected to be a regionaround whatever object 38 is closest to the range sensor 14. Theselected three-dimensional region of interest (x₂*, y₂*, z₂*) orvolume's position, size, and shape may all be functions of rangedistance 36 (d), rather than just being fixed volumes in space. Theselected three-dimensional region of interest (x₂*, y₂*, z₂*) mayinclude only objects 38 (see FIG. 2) in the scene 22 that are less thana minimum distance from the range sensor.

In one embodiment, the selected three-dimensional region of interest(x₂*, y₂*, z₂*) includes only objects 38 in the scene 22 within or onthe surface of a predefined three-dimensional volume (such as volume 312shown in FIG. 3A and described below). The three-dimensional volume maybe a cube, cylinder, rectangular prism, cone, triangular prism or anyother regular or irregular three-dimensional shape. Thethree-dimensional volume defines selected points. In one example, thespatial location of each of the selected points in the three-dimensionalvolume may be a function of time such that the position, size or shapeof the three-dimensional volume may change over time. This allows, forexample, tracking of a movable object (such as object-of-interest 304shown in FIGS. 3A and 3B).

In step 110 of FIG. 2, the controller 40 is configured to transform thethree-dimensional region of interest (x₂*, y₂*, z₂*) in the range sensorcoordinate system (R) to a three-dimensional region of interest (x₁*,y₁*, z₁*) in the first camera coordinate system (C₁). Referring to FIG.1, the three-dimensional regions of interest (x₂*, y₂*, z₂*) and (x₁*,y₁*, z₁*) are identical in an absolute space coordinate system A. Step110 involves transforming the three-dimensional region of interest fromone coordinate frame to another. Step 110 may include a sub-step 110A toobtain a first transformation matrix (T₂₁) for converting the rangesensor coordinate system (R) to the first camera coordinate system (C₁)such that:

${T_{21}\begin{bmatrix}{x\; 2} \\{y\; 2} \\{z\; 2}\end{bmatrix}} = \begin{bmatrix}{x\; 1} \\{y\; 1} \\{z\; 1}\end{bmatrix}$

The first transformation matrix (T₂₁) may be determined from the knownspatial or geometric relationship between the first camera 12 and therange sensor 14. As is known to those skilled in the art, given twoframes in three-dimensional space, it is possible to develop atransformation matrix that converts the coordinates from one frame tothe coordinates of another if the geometric relationship between the twoframes is known. The first camera 12 and the range sensor 14 may bepositioned such that the range sensor coordinate system (R) and thefirst camera coordinate system (C₁) involve a simple translation offrames [such as (x₂, y₂, z₂) to (x₁, y₁, z₁)]. In one example, where therange sensor and first camera coordinate systems are related by adisplacement along the y-axis of negative 5 units, the firsttransformation matrix (T₂₁) may be:

$T_{21} = \begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & {- 5} \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}$

Step 110 of FIG. 2 includes transforming the three-dimensional region ofinterest from the range sensor coordinate system (R) to the first cameracoordinate system (C₁) [(x₂*, y₂*, z₂*) to (x₁*, y₁*, z₁*)] using thefirst transformation (T₂₁) such that:

${T_{21}\begin{bmatrix}{x\; 2*} \\{y\; 2*} \\{z\; 2*}\end{bmatrix}} = \begin{bmatrix}{x\; 1*} \\{y\; 1*} \\{z\; 1*}\end{bmatrix}$

In step 112 of FIG. 2, the controller 40 is configured to map thethree-dimensional region of interest (x₁*, y₁*, z₁*) onto the firstimage plane 26 to obtain the two-dimensional region of interest (u₁*,v₁*). Step 112 may include a sub-step 112A to obtain a projection matrix(P₁) for projecting the first camera coordinate system (C₁) to the firstimage plane 26 such that:

${P_{1}\begin{bmatrix}{x\; 1} \\{y\; 1} \\{z\; 1}\end{bmatrix}} = {\begin{bmatrix}{u\; 1} \\{v\; 1}\end{bmatrix}.}$

Step 112 for mapping the three-dimensional region of interest (x₁*, y₁*,z₁*) onto the first image plane 26 to obtain the two-dimensional regionof interest (u₁*, v₁*) may be carried out using the projection matrix(P₁) such that:

${P_{1}\begin{bmatrix}{x\; 1*} \\{y\; 1*} \\{z\; 1*}\end{bmatrix}} = \begin{bmatrix}{u\; 1*} \\{v\; 1*}\end{bmatrix}$

Referring to FIG. 2, a second camera 50 (shown in FIG. 1) may be addedwith optional steps 114 to 118. The second camera 50 is spaced a seconddistance 52 from the range sensor 14 and defines a second cameracoordinate system (C₂). In this example, in step 114 (after theacquiring of the range image 30 in step 104) the controller 40 isconfigured to acquire a third image 54 of the scene 22. Referring toFIG. 1 and as noted above, the third image 54 is defined by a pluralityof third points 56 (u₃, v₃) in a third image plane 58.

In step 116 of FIG. 2, the controller 40 is configured to transform theplurality of third points 56 (u₃, v₃) from the third image plane 58 tothe first image plane 26. Step 116 may include a sub-step 116A todetermine a second transformation matrix (T₃₁) for converting the thirdimage plane 58 to the first image plane 26. The second transformationmatrix (T₃₁) may be determined from the known spatial or geometricrelationship between the second camera 50, the first camera 12 and therange sensor 14. In one example, where the first and second cameracoordinate systems are related by a displacement along the y-axis of 10units, the second transformation matrix (T₃₁) may be:

$T_{31} = \begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 10 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}$

In step 118 of FIG. 2 (prior to the conversion of the range image 30 tothe three-dimensional sample of points (x₂, y₂, z₂) in step 106) thecontroller 40 is configured to combine the third image 54 with the firstimage 20. Combining the images from the first and second cameras 12, 50improves resolution and allows for the use of low-resolution digitalcameras as the first and second cameras 12, 50. In one example, thefirst and second cameras 12, 50 are 5-megapixel color cameras.Additionally, other cameras may be employed with different overlappingfields of view, spectral sensitivity and/or high dynamic range, andunified through the process 100 above.

The process 100 of FIG. 2 may be employed for removal of backgroundclutter. FIG. 3A is a schematic diagram of an example first image 302 ofan object-of-interest 304, prior to implementation of the process 100.Referring to FIG. 3A, each of the circles 306 schematically representother objects in a scene that are less than a given distance, forexample 2 meters, from the first camera 12. Each of the triangles 308schematically represent other objects that are more than the givendistance from the first camera 12, i.e., background.

FIG. 3B is a schematic diagram of a segmented first image 310, whichrepresents the example first image 302 of FIG. 3A after implementationof the process 100. Here, the three-dimensional region of interest (x₂*,y₂*, z₂*) was selected to include only objects that are less than agiven distance from the first camera 12. Stated differently, FIG. 3Brepresents FIG. 3A after removal of all objects that are more than thegiven distance from the first camera 12. Alternatively, thethree-dimensional region of interest (x₂*, y₂*, z₂*) may be selected toinclude only objects in a scene within or on the surface of a predefinedthree-dimensional volume, such as volume 312 shown in FIG. 3A.

In summary, the data from the range sensor 14 is employed to select atwo-dimensional region of interest in the first image 20, which is atwo-dimensional intensity image. Referring to FIG. 1, the range sensor14 is calibrated to the same coordinate system as the first camera 12.The range image 30 is merged with the two-dimensional first image 20(color or monochrome) so that three-dimensional position data can beused as a criterion for selecting relevant parts of the first image 20.The range image 30 from the range sensor 14 is converted to athree-dimensional sample of points, which can be filtered to find onlypoints in a three-dimensional region of interest. The three-dimensionalregion of interest is then mapped back onto the two-dimensional firstimage plane 26 of the first camera 12, yielding a two-dimensional regionof interest in the first image 20.

This two-dimensional region of interest can then be processed byconventional computer vision techniques, while ignoring othernon-relevant parts of the two-dimensional first image 20. Stateddifferently, a range image 30 from the range sensor 14 is used tosegment a two-dimensional grayscale or color intensity image. Thisallows image segmentation that may be difficult or impossible withoutrange distance 36 (see FIG. 1) information. This process can beaccomplished with a low-resolution range sensor 14. The segmentation cansignificantly reduce the portion of the first image 20 that needs to beprocessed, thereby increasing the speed of the process.

The process 100 may be employed to segment a scene 22 where thestructural elements in the field of view are very similar and havechanging scale or random scale as a function of range distance 36 (d)such that typical techniques known to those skilled in the art are notsuitable. With the process 100, an object 38 at the target range may beeasily segmented for further analysis of internal features by thetwo-dimensional first and/or second cameras 12, 50. Additionally, theprocess 100 may be employed for range adaption where the segmentationtarget range is selected relative to the measurement of the nearestobject 38 (of a minimum size) in the field of view or the farthest thatcould be an object or background plane. Once the closest object range isfound, the segmentation may be done around that (this would segment theclosest object) or an object or plane could be selected relative to therear-most object or plane.

As noted above, the controller 40 of FIG. 1 may include a computingdevice that employs an operating system or processor 42 and memory 44for storing and executing computer-executable instructions.Computer-executable instructions may be compiled or interpreted fromcomputer programs created using a variety of programming languagesand/or technologies, including, without limitation, and either alone orin combination, Java™, C, C++, Visual Basic, Java Script, Perl, etc. Ingeneral, a processor 72 (e.g., a microprocessor) receives instructions,e.g., from a memory, a computer-readable medium, etc., and executesthese instructions, thereby performing one or more processes, includingone or more of the processes described herein. Such instructions andother data may be stored and transmitted using a variety ofcomputer-readable media.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory (e.g., tangible) medium thatparticipates in providing data (e.g., instructions) that may be read bya computer (e.g., by a processor of a computer). Such a medium may takemany forms, including, but not limited to, non-volatile media andvolatile media. Non-volatile media may include, for example, optical ormagnetic disks and other persistent memory. Volatile media may include,for example, dynamic random access memory (DRAM), which may constitute amain memory. Such instructions may be transmitted by one or moretransmission media, including coaxial cables, copper wire and fiberoptics, including the wires that comprise a system bus coupled to aprocessor of a computer. Some forms of computer-readable media include,for example, a floppy disk, a flexible disk, hard disk, magnetic tape,any other magnetic medium, a CD-ROM, DVD, any other optical medium,punch cards, paper tape, any other physical medium with patterns ofholes, a RAM, a PROM, an EPROM, a FLASH-EEPROM, any other memory chip orcartridge, or any other medium from which a computer can read.

The detailed description and the drawings or figures are supportive anddescriptive of the invention, but the scope of the invention is definedsolely by the claims. While some of the best modes and other embodimentsfor carrying out the claimed invention have been described in detail,various alternative designs and embodiments exist for practicing theinvention defined in the appended claims. Furthermore, the embodimentsshown in the drawings or the characteristics of various embodimentsmentioned in the present description are not necessarily to beunderstood as embodiments independent of each other. Rather, it ispossible that each of the characteristics described in one of theexamples of an embodiment can be combined with one or a plurality ofother desired characteristics from other embodiments, resulting in otherembodiments not described in words or by reference to the drawings.Accordingly, such other embodiments fall within the framework of thescope of the appended claims.

The invention claimed is:
 1. A system comprising: a first cameradefining a first camera coordinate system (C₁) and configured to acquirea first image of a scene; a range sensor spaced a first distance fromthe first camera and defining a range sensor coordinate system (R); acontroller operatively connected to the first camera and range sensor,the controller having a processor and a tangible, non-transitory memorydevice on which is recorded instructions for executing a method forobtaining a two-dimensional region of interest (u₁*, v₁*) in the firstimage; wherein execution of the instructions by the processor causes thecontroller to: acquire the first image of the scene with the firstcamera, the first image being represented by a plurality of first points(u₁, v₁) in a first image plane; acquire a range image of the scene withthe range sensor, the range image being represented by a plurality ofsecond points (u₂, v₂, d) in a second image plane; convert the rangeimage to a three-dimensional sample of points (x₂, y₂, z₂) in the rangesensor coordinate system (R); select a three-dimensional region ofinterest (x₂*, y₂*, z₂*) in the range sensor coordinate system (R) fromthe three-dimensional sample of points (x₂, y₂, z₂); wherein theselected three-dimensional region of interest (x₂*, y₂*, z₂*) in therange sensor coordinate system (R) includes only objects in the scene ona surface of or within a three-dimensional volume, the three-dimensionalvolume defining selected points; wherein a spatial location of each ofthe selected points is a function of time; wherein the selectedthree-dimensional region of interest (x₂*, y₂*, z₂*) in the range sensorcoordinate system (R) includes only objects in the scene that are lessthan a minimum distance from the range sensor; and wherein saidconverting the range image to a three-dimensional sample of points (x₂,y₂, z₂) in the range sensor coordinate system (R) includes: obtaining aconversion matrix (P₂) for converting the range sensor coordinate system(R) to the second image plane; and using an inverse of the conversionmatrix for converting the range image to the three-dimensional sample ofpoints (x₂, y₂, z₂).
 2. The system of claim 1, wherein: each of theplurality of second points (u₂, v₂, d) in the range image includes arange distance (d) corresponding to a respective distance from the rangesensor to the objects in the scene; the selected three-dimensionalregion of interest (x₂*, y₂*, z₂*) defines selected points, a spatiallocation of each of the selected points being a function of the rangedistance (d).
 3. The system of claim 1, wherein the first camera, thecontroller and the range sensor are part of a single device.
 4. Thesystem of claim 1, wherein the controller is further configured to:transform the three-dimensional region of interest from the range sensorcoordinate system (R) to the first camera coordinate system (C₁) [(x₂*,y₂*, z₂*) to (x₁*, y₁*, z₁*)]; and map the three-dimensional region ofinterest (x₁*, y₁*, z₁*) in the first camera coordinate system (C₁) ontothe first image plane to obtain the two-dimensional region of interest(u₁*, v₁*).
 5. The system of claim 4, wherein said transforming thethree-dimensional region of interest from the range sensor coordinatesystem (R) to the first camera coordinate system (C₁) [(x₂*, y₂*, z₂*)to (x₁*, y₁*, z₁*)] includes: obtaining a first transformation matrix(T₂₁) for converting the range sensor coordinate system (R) to the firstcamera coordinate system (C₁).
 6. The system of claim 4, wherein saidmapping the three-dimensional region of interest (x₁*, y₁*, z₁*) in thefirst camera coordinate system (C₁) onto the first image plane includes:obtaining a projection matrix (P₁) for projecting the first cameracoordinate system (C₁) to the first image plane.
 7. The system of claim4, further comprising: a second camera spaced a second distance from therange sensor and defining a second camera coordinate system (C₂);wherein the controller is configured to: after said acquiring the rangeimage, acquire a third image of the scene, the third image being definedby a plurality of third points (u₃, v₃) in a third image plane;determine a second transformation matrix (T₃₁) for converting the thirdimage plane to the first image plane; transform the plurality of thirdpoints (u₃, v₃) from the third image plane to the first image plane viathe second transformation matrix (T₃₁); and prior to the conversion ofthe range image to the three-dimensional sample of points (x₂, y₂, z₂),combine the third image with the first image.
 8. A method of obtaining atwo-dimensional region of interest in a two-dimensional image, themethod comprising: acquiring a first image of a scene with a firstcamera, the first image being represented by a plurality of first points(u₁, v₁) in a first image plane, the first camera defining a firstcamera coordinate system (C₁); acquire a range image of the scene with arange sensor, the range image being represented by a plurality of secondpoints (u₂, v₂, d) in a second image plane; wherein the range sensor isspaced a first distance from the first camera and defines a range sensorcoordinate system (R); converting the range image to a three-dimensionalsample of points (x₂, y₂, z₂) in the range sensor coordinate system (R);selecting a three-dimensional region of interest (x₂*, y₂*, z₂*) in therange sensor coordinate system (R) from the three-dimensional sample ofpoints (x₂, y₂, z₂); transforming the three-dimensional region ofinterest from the range sensor coordinate system (R) to the first cameracoordinate system (C₁) [(x₂*, y₂*, z₂*) to (x₁*, y₁*, z₁*)]; mapping thethree-dimensional region of interest (x₁*, y₁*, z₁*) in the first cameracoordinate system (C₁) onto the first image plane to obtain thetwo-dimensional region of interest (u₁*, v₁*); wherein the selectedthree-dimensional region of interest (x₂*, y₂*, z₂*) in the range sensorcoordinate system (R) includes only objects in the scene on a surface ofor within a three-dimensional volume, the three-dimensional volumedefining selected points; a spatial location of each of the selectedpoints is a function of time; wherein the selected three-dimensionalregion of interest (x₂*, y₂*, z₂*) in the range sensor coordinate system(R) includes only objects in the scene that are less than a minimumdistance from the range sensor; and wherein said converting the rangeimage to a three-dimensional sample of points (x₂, y₂, z₂) in the rangesensor coordinate system (R) includes: obtaining a conversion matrix(P₂) for converting the range sensor coordinate system (R) to the secondimage plane; and using an inverse of the conversion matrix forconverting the range image to the three-dimensional sample of points(x₂, y₂, z₂).
 9. The method of claim 8, wherein said transforming thethree-dimensional region of interest from the range sensor coordinatesystem (R) to the first camera coordinate system (C₁) [(x₂*, y₂*, z₂*)to (x₁*, y₁*, z₁*)] includes: obtaining a first transformation matrix(T₂₁) for converting the range sensor coordinate system (R) to the firstcamera coordinate system (C₁).
 10. The method of claim 8, wherein saidmapping the three-dimensional region of interest (x₁*, y₁*, z₁*) in thefirst camera coordinate system (C₁) onto the first image plane includes:obtaining a projection matrix (P₁) for projecting the first cameracoordinate system (C₁) to the first image plane.
 11. The method of claim8, further comprising: after said acquiring the second image, acquiringa third image of the scene with a second camera, the third image beingdefined by a plurality of third points (u₃, v₃) in a third image plane;wherein the second camera is spaced a second distance from the rangesensor and defines a second camera coordinate system (C₂); determining asecond transformation matrix (T₃₁) for converting the third image planeto the first image plane; transforming the plurality of third points(u₃, v₃) from the third image plane to the first image plane via thesecond transformation matrix (T₃₁); and prior to the conversion of therange image to the three-dimensional sample of points (x₂, y₂, z₂),combining the third image with the first image.